System and method for eye-tracking

ABSTRACT

A system for eye-tracking according to an embodiment of the present invention includes a data collection unit that acquires face information of a user and location information of the user from an image captured by a photographing device installed at each of one or more points set within a three-dimensional space and an eye tracking unit that estimates a location of an area gazed at by the user in the three-dimensional space from the face information and the location information, and maps spatial coordinates corresponding to the location of the area to a three-dimensional map corresponding to the three-dimensional space.

CROSS REFERENCE TO RELATED APPLICATIONS AND CLAIM OF PRIORITY

This application claims benefit under 35 U.S.C. 119(e), 120, 121, or365(c), and is a National Stage entry from International Application No.PCT/KR2019/004229, filed Apr. 9, 2019, which claims priority to thebenefit of Korean Patent Application Nos. 10-2018-0041921 filed on Apr.11, 2018 and 10-2019-0040684 filed on Apr. 8, 2019 in the KoreanIntellectual Property Office, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

Embodiments of the present invention relate to an eye trackingtechnology.

BACKGROUND ART

Eye tracking is a technology that detects a user's eyeball movement totrack a location of eyes, and methods such as an image analysis method,a contact lens method, and a sensor attachment method may be used. Theimage analysis method detects a pupil movement through an analysis ofreal-time camera image and calculates a direction of eyes based on afixed location reflected on a cornea. The contact lens method usesreflected light from a mirror built-in contact lens, a magnetic field ofa coil built-in contact lens, or the like, and is less convenient, buthas higher accuracy. The sensor attachment method attaches a sensoraround the eyes to detect an eyeball movement using a change in anelectric field according to an eye movement, and can detect the eyeballmovement even when the eyes are closed (sleep etc.).

In recent years, devices and fields targeted for application of an eyetracking technology are gradually expanding, and accordingly, attemptsto utilize the eye tracking technology in collecting data such as apreferred product or service by tracking people's eyes are increasing.

However, the conventional eye gaze tracking technology has been limitedto tracking user's eyes on a two-dimensional screen, and accordingly,there is a limit in providing various services through user's eyetracking in a three-dimensional space

SUMMARY Technical Problem

Embodiments of the present invention are to more accurately track user'seyes in a three-dimensional space.

According to an exemplary embodiment of the invention, there is provideda system for eye-tracking including: a data collection unit thatacquires face information of a user and location information of the userfrom an image captured by a photographing device installed at each ofone or more points set within a three-dimensional space; and an eyetracking unit that estimates a location of an area gazed at by the userin the three-dimensional space from the face information and thelocation information, and maps spatial coordinates corresponding to thelocation of the area to a three-dimensional map corresponding to thethree-dimensional space.

The data collection unit may acquire the location information of theuser by using one or more of location coordinates of the photographingdevice, a location or size of the user in the image, and a distancebetween the photographing device and a terminal possessed by the user.

The face information may include one or more of a face location, pupillocation, face vector, and pupil vector of the user.

The eye tracking unit may determine a location of an area correspondingto the location information in the three-dimensional space, grasp apoint corresponding to the face location or the pupil location on thedetermined location of the area, and predict, as the location of thearea gazed at by the user, a location of an object disposed in adirection in which the face vector or the pupil vector is directed fromthe grasped point.

Each of the photographing devices may be installed at different pointsin the three-dimensional space to capture the image.

The data collection unit may predict a movement trajectory of the userfrom the face information and the location information acquired fromeach image, and the eye tracking unit may predict an eye trajectory ofthe user from the movement trajectory, and the face information and thelocation information acquired from each image, and map spatialcoordinates corresponding to the eye trajectory to a three-dimensionalmap corresponding to the three-dimensional space.

According to another embodiment of the present invention, there isprovided a system for eye-tracking including: a data collection unitthat acquires at least one of face information of a user, locationinformation of a user, user number information, and human bodyinformation of a user from an image captured by a photographing deviceinstalled at each of one or more points set within a three-dimensionalspace; an eye tracking unit that estimates an eye gaze location of acorresponding user from the face information of the user and thelocation information of the user in the three-dimensional space; and aneye-related analysis unit that analyzes eye-related contents of the userbased on one or more of the user number information, the human bodyinformation of the user, and eye gaze location information of the userare included.

The human body information of the user may include one or more ofgender, age, race, and emotion of the user, and the eye-related analysisunit may analyze eye dwelling time individually for one or more of anage, gender, race, and emotion of a user for a predetermined object in athree-dimensional space based on the human body information of the userand the eye gaze location information of the user.

The eye-related analysis unit may analyze a ratio of a person who lookedat a predetermined object to a floating population for the object in thethree-dimensional space based on the user number information and theuser eye gaze location information.

The eye-related analysis unit may determine to change the object toanother object or to change a location of the object in thethree-dimensional space when the ratio of the person who looked at theobject to the floating population is less than or equal to a presetratio for the object.

The eye-related analysis unit may analyze an eye distribution degree foreach object in the three-dimensional space based on the user numberinformation and the user eye gaze location information.

The user human body information may include one or more of gender, age,race, and emotion of the user, and the eye-related analysis unit maydetermine whether there is a harmful object at a point corresponding toa location gazed at by the corresponding user or whether the point is apreset prohibited zone or a dangerous zone, using one or more of thehuman body information of the user as a reference.

The user human body information may include one or more of the gender,age, race, and emotion of the user, and the eye-related analysis unitmay extract one or more of eye dwelling time and the number of gazingeyes for a preset object or a preset zone of a corresponding user basedon the user eye gaze location information and determine a risk index ofa corresponding user based on one or more of the human body informationof the user, the eye dwelling time, and the number of gazing eyes.

According to an exemplary embodiment of the invention, there is provideda method for eye-tracking including: acquiring face information of auser and location information of the user from an image captured by aphotographing device installed at each of one or more points set withina three-dimensional space; estimating a location of an area gazed at bythe user in the three-dimensional space from the face information andthe location information, and mapping spatial coordinates correspondingto the location of the area to a three-dimensional map corresponding tothe three-dimensional space.

In the acquiring, the location information of the user may be acquiredby using one or more of location coordinates of the photographingdevice, a location or size of the user in the image, and a distancebetween the photographing device and a terminal possessed by the user.

The face information may include one or more of a face location, pupillocation, face vector, and pupil vector of the user.

The estimating may include determining a location of an areacorresponding to the location information in the three-dimensionalspace, grasping a point corresponding to the face location or the pupillocation on the determined location of the area, and predicting, as thelocation of the area gazed at by the user, a location of an objectdisposed in a direction in which the face vector or the pupil vector isdirected from the grasped point.

Each of the photographing devices may be installed at different pointsin the three-dimensional space to capture the image.

In the acquiring, a movement trajectory of the user may be predictedfrom the face information and the location information acquired fromeach image is predicted, in the estimating, an eye trajectory of theuser may be predicted from the movement trajectory, and the faceinformation and the location information acquired from each image, andin the mapping, spatial coordinates corresponding to the eye trajectorymay be mapped to a three-dimensional map corresponding to thethree-dimensional space.

According to embodiments of the present invention, face information of auser and location information of the user are acquired from an imagecaptured by one or more photographing devices and the location of anarea gazed at by the user in the three-dimensional space therefrom,thereby capable of tracking eyes of the user more accurately in thethree-dimensional space.

Further, according to an embodiment of the present invention, the faceinformation of the user and the location information of the userphotographed by one or more photographing devices are acquired and amovement trajectory and eye trajectory of the user are predictedtherefrom, thereby capable of grasping eye movement of the user moreaccurately in the three-dimensional space.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a detailed configuration of asystem for eye-tracking according to an embodiment of the presentinvention.

FIG. 2 is an example for illustrating a process of acquiring locationinformation of a user by a data collection unit according to anembodiment of the present invention.

FIG. 3 is another example for illustrating a process of acquiring thelocation information of the user by the data collection unit accordingto an embodiment of the present invention.

FIG. 4 is an example for illustrating a process of predicting a locationof an area gazed at by a user according to an embodiment of the presentinvention.

FIG. 5 is a flowchart illustrating a method for eye-tracking accordingto an embodiment of the present invention.

FIG. 6 is a diagram illustrating a configuration of a system foreye-tracking according to another embodiment of the present invention.

FIG. 7 is a block diagram for exemplarily illustrating a computingenvironment including a computing device suitable for use in exemplaryembodiments.

DETAILED DESCRIPTION

Hereinafter, specific embodiments of the present invention will bedescribed with reference to the accompanying drawings. The followingdetailed description is provided to aid in a comprehensive understandingof a method, a device and/or a system described in the presentspecification. However, the detailed description is only forillustrative purpose and the present invention is not limited thereto.

In describing the embodiments of the present invention, when it isdetermined that a detailed description of known technology related tothe present invention may unnecessarily obscure the gist of the presentinvention, the detailed description thereof will be omitted. Inaddition, terms to be described later are terms defined in considerationof functions in the present invention, which may vary depending onintention or custom of a user or operator. Therefore, the definition ofthese terms should be made based on the contents throughout thisspecification. The terms used in the detailed description are only fordescribing the embodiments of the present invention and should not beused in a limiting sense. Unless expressly used otherwise, a singularform includes a plural form. In this description, expressions such as“including” or “comprising” are intended to indicate any property,number, step, element, and some or combinations thereof, and suchexpressions should not be interpreted to exclude the presence orpossibility of one or more other properties, numbers, steps, elementsother than those described, and some or combinations thereof.

FIG. 1 is a block diagram illustrating a detailed configuration of asystem for eye-tracking 100 according to one embodiment of the presentinvention.

Referring to FIG. 1, the system for eye-tracking 100 according to oneembodiment of the present invention includes an eye tracking device 102and a photographing device 104.

The eye tracking device 102 is communicatively connected to thephotographing device 104 through a communication network. In someembodiments, a communication network 150 may include the Internet, oneor more local area networks, wide area networks, cellular networks,mobile networks, other types of networks, or a combination of thesenetworks.

The eye tracking device 102 may acquire one or more of face informationof a user, location information of a user, user number information, andhuman body information a user from a captured of the photographingdevice 104. Here, the photographing device 104 may be installed at eachof one or more points set in the three-dimensional space.

The photographing device 104 may be, for example, a camera, a camcorder,a closed circuit television (CCTV), etc. In an exemplary embodiment, aninfrared camera may be used as the photographing device 104, but is notlimited thereto. The photographing device 104 may be installed on, forexample, a ceiling or structure in a building, an upper end portion of akiosk, inside an elevator, inside a vehicle, a display stand, a concert,a lecture hall, a hypermarket, a shopping mall, a shopping mall, arestaurant, an airport, a subway, a children's house, a casino, a seniorclub, a factory, etc. Further, the photographing devices 104 may berespectively installed at different points in the three-dimensionalspace, and installation angles thereof may also be different. As anexample, the photographing device 104 may be installed at each of one ormore points set in the hypermarket to capture images of users of thehypermarket, and transmit the captured images to the eye tracking device102.

The eye tracking device 102 may include a data collection unit 110, aneye tracking unit 120, and an eye-related analysis unit 130. In oneembodiment, the data collection unit 110, the eye tracking unit 120, andthe eye-related analysis unit 130 are implemented using one or morephysically separated devices, or implemented by one or more processorsor a combination of one or more processors and software, and may not beclearly distinguished in specific operations unlike the illustratedexample.

The data collection unit 110 may acquire one or more of the faceinformation of the user, the location information of the user, usernumber information, and the human body information of the user from animage captured by the photographing device 104.

Specifically, the data collection unit 110 may extract a face area andeye area of a user from the image using various rule-based algorithms ordeep learning models, and acquire face information of the user from theextracted face area and eye area.

Here, the face information of the user may include one or more of a facelocation, face size, pupil location, face vector, and pupil vector ofthe corresponding user. Here, the rule-based algorithm is an algorithmacquire various data for eye tracking using a predetermined imageprocessing technique, an image processing technique, or a mathematicalexpression, and may be, for example, face recognition algorithms (e.g.,principal component analysis (PCA), linear discriminant analysis (LDA),etc.), face's feature point detection algorithms (e.g., support vectormachine (SVM), speeded up robust features (SURF), etc.), an image-basedhead-tracking algorithm, an pupil extraction and pupil locationcoordinates calculation algorithm. Further, the deep learning model maybe, for example, a convolutional neural network (CNN) model. The datacollection unit 110 may acquire face information of the user, such asthe face location, pupil location, face vector, and pupil vector of theuser from a captured image using various rule-based algorithms, deeplearning models, etc.

In an exemplary embodiment, the data collection unit 110 may recognize aperson from the captured image. That is, the data collection unit 110may recognize a person in the captured image and detect an area wherethe person is located. For example, the data collection unit 110 mayrecognize a person in the captured image and detect then area where theperson is located using Fast area convolutional neural networks (RCNN)or Mask RCNN technique.

Next, the data collection unit 110 may check the number of people (thenumber) in the area where the person is located. The data collectionunit 110 may receive the captured image periodically or in real timefrom the photographing device 104, and check the number of people fromeach captured image to analyze hourly floating population in thecorresponding place (the place where the photographing device 104 isinstalled). The user number information may include the number of peopleincluded in each captured image and the hourly floating population inthe corresponding place.

Further, the data collection unit 110 may recognize the face area of theperson in the area where the person is located. The data collection unit110 may acquire user human body information including one or more ofgender, age, race, and emotion of the corresponding user based on theface area of the person. For example, the data collection unit 110 mayacquire the user human body information including one or more of thegender, age, race, and emotion of the user based on the face area of theperson using various rule-based algorithms or deep learning models.

Further, the data collection unit 110 may recognize a location of a mainpoint (i.e., a landmark) (e.g., eyes, nose, and mouth) in the facewithin the face area of the person. The data collection unit 110 mayextract a face size and face vector of the corresponding user based on aface area image of the person from among the captured image.

Further, the data collection unit 110 may detect an eye area based on alocation of an eye point in the face. The data collection unit 110 maydetect the pupil vector (eye vector) of the user based on an eye areaimage.

Further, the data collection unit 110 may acquire the locationinformation of the user using one or more of location coordinates of thephotographing device 104, a location of the user in the captured image,a size (or face size) of the user in the captured image, and a distancebetween the photographing device 104 and a terminal possessed by theuser.

FIGS. 2 and 3 are examples for illustrating a process of acquiringlocation information of a user by the data collection unit 110 accordingto an embodiment of the present invention.

As an example, referring to FIG. 2, when it is assumed that the user islocated at an X point (a, b) and the photographing device 104 is locatedat a Y point (c, d), the data collection unit 110 may grasp the locationand size of the user in the captured image and acquire the locationinformation of the user (i.e., location coordinates (a, b) of the Xpoint) therefrom. The data collection unit 110 may acquire the locationinformation of the user based on an installation location andinstallation angle of the photographing device 104 and the location andsize of the user in the captured image. In this case, since theinstallation location and angle of the photographing device 104 arefixed, the data collection unit 110 may be provided with locationcoordinates for each point in the image in advance. Further, the datacollection unit 110 may predict a distance R between the photographingdevice 104 and the user according to the user size at each point in theimage, and correct the location information of the user.

As another example, referring to FIG. 3, the user may move around with aterminal 301 such as a smartphone in a three-dimensional space. In thiscase, the terminal 301 and the photographing device 104 may be providedwith a wireless communication module for signal transmission andreception, for example, a Bluetooth module, a Wi-Fi module, etc., andmay transmit and receive a signal through the communication module. Thedata collection unit 110 may calculate a distance R′ between thephotographing device 104 and the terminal 301 through a signal exchangedbetween the photographing device 104 and the terminal 301. Further, thedata collection unit 110 may grasp one or more candidate pointscorresponding to the location of the user in the captured image, anddetermine a location of a candidate point, which is separated by thedistance R′ from the location coordinates of the photographing device104, among the candidate points, as a location of the user.

As another example, the data collection unit 110 may acquire locationinformation of the user by receiving the location coordinates of theterminal 301 acquired through a GPS module of the terminal 301 possessedby the user. As such, the data collection unit 110 may acquire thelocation information of the user in various ways.

Returning to FIG. 1 again, the eye tracking unit 120 tracks a locationof an area gazed at by the user in the three-dimensional space from theface information and location information of the user acquired by thedata collection unit 110.

Specifically, the eye tracking unit 120 may determine a location (i.e.,location of the user) corresponding to the location information of theuser in the three-dimensional space, and grasp the face location orpupil location of the user on the location of user. Next, the eyetracking unit 120 may estimate, as the location of the area gazed at bythe user, a location of an object disposed in a direction in which theface vector or pupil vector of the user faces is directed from the facelocation or pupil location of the user.

FIG. 4 is an example for illustrating a process of estimating a locationof an area gazed at by a user according to one embodiment of the presentinvention.

As an example, referring to FIG. 4, the eye tracking unit 120 maydetermine location information of the user acquired by the datacollection unit 110, that is, a point X corresponding to the locationcoordinates (a, b) as an area where the user is located and grasp apoint Z corresponding to the face location or pupil location of the useron the X point, that is, three-dimensional spatial coordinates (a, b,p).

Further, the eye tracking unit 120 may estimate, as the location of thearea gazed at by the user, a location of an object A disposed in adirection in which the face vector or pupil vector of the user isdirected from the grasped point Z, that is, three-dimensional spatialcoordinates (1, m, n). Here, it is assumed that the eye tracking unit120 is provided with three-dimensional spatial coordinates of eachobject disposed in the three-dimensional space in advance.

Further, the data collection unit described above may acquire a movementtrajectory of the user from the face information and locationinformation acquired from each image captured by the photographingdevices 104 respectively installed at different points in thethree-dimensional space. As an example, when an image of a user issequentially captured at points A and B, the data collection unit 110may estimate that the user has moved from point A to point B using faceinformation and location information of the user acquired from the imagecaptured at point A and face information and location information of theuser acquired from the image captured at point B.

Further, the eye tracking unit 120 may track an eye trajectory of theuser from the movement trajectory acquired by the data collection unit110, and the face information of the user and the location informationof the user acquired from each image captured by the plurality ofphotographing devices 104. Specifically, since the face location, pupillocation, face vector, and pupil vector of the user, and the locationinformation of the user changes each time the user moves, the location(user eye gaze location) of the area gazed at by the use can beestimated by tracking the eye trajectory of the user from the faceinformation and the location information at each location according tothe movement trajectory of the user.

Further, the eye tracking unit 120 maps spatial coordinatescorresponding to a location of a place gazed at by the user in thethree-dimensional space to a three-dimensional map corresponding to thecorresponding three-dimensional space. Here, the three dimensional mapis a map acquired by modeling the three-dimensional space, and may beprovided with three dimensional spatial coordinates for each point.

Further, the eye tracking unit 120 may map spatial coordinatescorresponding to the eye trajectory of the user to the three-dimensionalmap corresponding to the three-dimensional space. Specifically, the eyetracking unit 120 may track the eye trajectory of the user bysequentially mapping spatial coordinates corresponding to the locationof the area gazed at by the user in the three-dimensional space to avirtual three dimensional map, and accordingly acquire and accumulatevarious data according to eye tracking from the user. In this case, whendata acquired by eye tracking of a plurality of users is accumulated,big data may be formed, and the big data may be utilized in variousservices.

As an example, the eye tracking unit 120 may acquire data for each userby tracking the eyes of a plurality of users who have visited ahypermarket. In this case, data for each of the users may be accumulatedto form big data related to a product preferred by visitors of thehypermarket, and the big data can be used as marketing information, suchas increasing inventory for the preferred product or disposing thepreferred product in a place well visible to people's eye. As anotherexample, the eye tracking unit 120 may acquire data for each student bytracking the eyes of a plurality of students in a classroom during thecourse of a lecture. In this case, data for each of the students may beaccumulated to form big data related to concentration of a lecture ofstudents who listen to the lecture, and the big data may be used aslecture materials to increase concentration of a class.

The eye-related analysis unit 130 may analyze contents related to eyesof the user based on data acquired by the data collection unit 110 andthe eye tracking unit 120. In an exemplary embodiment, the eye-relatedanalysis unit 130 may analyze an eye dwelling time of the user for anobject disposed in a three-dimensional space. That is, the eye-relatedanalysis unit 130 may analyze the eye dwelling time of the user for anobject based on information on a location (i.e., the location of thearea gazed at by the user) of the object disposed in the direction inwhich the face vector or pupil vector of the user tracked by the eyetracking unit 120 is directed. In this time, the eye-related analysisunit 130 may analyze, which part of the object has been viewed and forhow long for each user.

Further, the eye-related analysis unit 130 may analyze the eye dwellingtime individually for one or more of the age, gender, race, and emotionof the user for a predetermined object in a three-dimensional spacebased on the human body information of the user and the user eye gazelocation information.

Further, the eye-related analysis unit may analyze a ratio of a personwho looked at a predetermined object to a floating population for theobject in the three-dimensional space based on the user numberinformation and the user eye gaze location information.

For example, when the object is a product on the display stand, anelevator advertisement, a taxi advertisement, an outdoor screenadvertisement, etc., the eye-related analysis unit 130 may analyze theeye dwelling time of an object individually for each user (includingindividually for each use age, gender, race, emotion, etc.), and mayanalyze a ratio of a person who looked at the object to a floatingpopulation.

In this case, for the object, the eye-related analysis unit maydetermine to change the object to another object or to change a locationof the object in the three-dimensional space when the ratio of theperson who looked at the object to the floating population is less thanor equal to a preset ratio.

Further, the eye-related analysis unit may analyze an eye distributiondegree for each object in the three-dimensional space based on the usernumber information and the user eye gaze location information. That is,the eye-related analysis unit 130 may analyze information on whatproportions of people present in the three-dimensional space are lookingat each object. When such information is used, for example, it ispossible to rearrange an arrangement of products on display stands bygiving priority to the product with high interest of people. Further,since the eye distribution degree of people can be checked for eachadvertisement (i.e., object) location, it is also possible to readjustthe advertisement location. The information analyzed by the eye-relatedanalysis unit 130 may also be used to adjust the location of a milestoneor warning message.

Further, the eye-related analysis unit 130 may extract the object havingthe highest eye distribution degree based on the eye distribution degreefor the object in the three-dimensional space. When such information isused, for example, it is possible to give an effect such as a highlightto an object having the highest eye distribution degree in a lecture orperformance.

Further, the eye-related analysis unit may determine whether the pointcorresponding to the location gazed at by the corresponding user is aharmful object (or a prohibited zone or a dangerous zone) using the age,gender, race, etc. of the user as a reference based on the user humanbody information and the user eye gaze location information. Forexample, the eye-related analysis unit 130 may determine whether theuser is a child under 7 years of age, and whether the object that theuser is looking at is a harmful object such as a knife. Further, theeye-related analysis unit 130 may take a measure (e.g., beep soundgeneration) to induce the eyes of the corresponding user to anotherplace or to give a warning.

Further, the eye-related analysis unit may analyze a risk index of acorresponding user based on the user human body information and the usereye gaze position information. In an exemplary embodiment, when the usergaze location information corresponds to the installation location (oraccess restricted area, etc.) of the photographing device 104, theeye-related analysis unit 130 may check one or more of the eye dwellingtime of the user and the number of gazing eyes to calculate the riskindex of the user. For example, when a user of an adult male gazes thephotographing device 104 for a predetermined time or more, orcontinuously watches for a predetermined number of times or more, theeye-related analysis unit 130 may determine that the risk index of theuser is high.

Here, the eye-related analysis unit 130 may assign a user risk indexscore for each age, gender, race, and emotion in the user human bodyinformation. The eye-related analysis unit 130 may give a higher riskindex score of the user as the user eye dwelling time and number ofgazing eyes for a preset object (e.g., a photographing device, etc.) ora preset zone (e.g., a restricted access zone) increases. In this case,it is possible to prevent terrorism by extracting a person at high riskof terrorism in public places such as an airports or subways.

FIG. 5 is a flowchart illustrating a method for eye-tracking accordingto one embodiment of the present invention. The method illustrated inFIG. 5 may be performed, for example, by the system for eye-tracking 100illustrated in FIG. 1.

First, the data collection unit 110 acquires face information of a userand location information of the user from an image captured by thephotographing device installed at each of one or more points set in thethree-dimensional space (S510). In this case, the face information ofthe user may include one or more of face location, pupil location, facevector, and pupil vector of the user. Further, each of the photographingdevices may be installed at different points in the three-dimensionalspace to capture an image.

Next, the eye tracking unit 120 predicts a location of an area gazed atby the user in the three-dimensional space from the face information andthe location information (S520).

Next, the eye tracking unit 120 maps spatial coordinates correspondingto the location of the area on a three dimensional map corresponding tothe three-dimensional space (S530).

Meanwhile, in the flowchart illustrated in FIG. 5, the method or processis described as being divided into a plurality of steps, but at leastsome of the steps may be performed by changing the order, may beperformed in combination with other steps, may be omitted, may beperformed by being divided into detailed steps, or may be performed byadding one or more steps (not illustrated) thereto.

FIG. 6 is a diagram illustrating the configuration of the system foreye-tracking 100 according to another embodiment of the presentinvention. Here, the parts different from the embodiment illustrated inFIG. 1 will be mainly described.

Referring to FIG. 6, an eye tracking device 102 may be integrallyinstalled at the same place as the photographing device 104. In anexemplary embodiment, the eye tracking device 102 and the photographingdevice 104 may be functionally distinguished from each other, but may beimplemented by being physically integrated with each other.

Each eye tracking device 102 may transmit data (for example, faceinformation of a user, location information of a user, user numberinformation, and human body information of a user, location of an areagazed at by a user, spatial coordinates of a point gazed at by a user,and eye gaze-related analysis information, etc.) processed by the eyetracking device 102 to a server 140. However, the present invention isnot limited thereto, and some configurations (for example, eye-relatedanalysis unit 130) of the eye tracking device 102 may be implemented inthe server 140.

FIG. 7 is a block diagram illustrating and exemplifying a computingenvironment 10 that includes a computing device suitable for use in theexemplary embodiment. In the illustrated embodiment, each component mayhave different functions and capabilities in addition to those describedbelow, and additional components may be included in addition to thosedescribed below.

The illustrated computing environment 10 includes a computing device 12.In one embodiment, the computing device 12 may be the system foreye-tracking 100 or one or more components included in the system foreye-tracking 100.

The computing device 12 includes at least one processor 14, acomputer-readable storage medium 16, and a communication bus 18. Theprocessor 14 may cause the computing device 12 to perform stepsaccording to the exemplary embodiment described above. For example, theprocessor 14 may execute one or more programs stored on thecomputer-readable storage medium 16. The one or more programs mayinclude one or more computer-executable instructions, which, whenexecuted by the processor 14, may be configured to cause the computingdevice 12 to perform steps according to the exemplary embodiment.

The computer-readable storage medium 16 is configured to store thecomputer-executable instruction or program code, program data, and/orother suitable forms of information. A program 20 stored in thecomputer-readable storage medium 16 includes a set of instructionsexecutable by the processor 14. In one embodiment, the computer-readablestorage medium 16 may be a memory (volatile memory such as a randomaccess memory, non-volatile memory, or any suitable combinationthereof), one or more magnetic disk storage devices, optical diskstorage devices, flash Memory devices, other types of storage media thatare accessible by the computing device 12 and store desired information,or any suitable combination thereof.

The communication bus 18 interconnects various other components of thecomputing device 12, including the processor 14 and thecomputer-readable storage medium 16.

The computing device 12 may also include one or more input/outputinterfaces 22 that provide an interface for one or more input/outputdevices 24, and one or more network communication interfaces 26. Theinput/output interface 22 may include a scroll screen, an inputinterface, and an input screen. The input/output interface 22 and thenetwork communication interface 26 are connected to the communicationbus 18. The input/output device 24 may be connected to other componentsof the computing device 12 through the input/output interface 22. Theexemplary input/output device 24 may include a pointing device (such asa mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touch screen), a voice or sound input device, input devices suchas various types of sensor devices and/or photographing devices, and/oroutput devices such as a display devices, a printer, a speaker, and/or anetwork card. The exemplary input/output device 24 may be includedinside the computing device 12 as a component constituting the computingdevice 12, or may be connected to the computing device 12 as a separatedevice distinct from the computing device 12.

Although the exemplary embodiment of the present invention has beendescribed in detail as above, those skilled in the art to which thepresent invention pertains will understand that various modificationsmay be made thereto within the limit that do not depart from the scopeof the present invention. Therefore, the scope of rights of the presentinvention should not be limited to the described embodiments, but shouldbe defined not only by claims set forth below but also by equivalents ofthe claims.

What is claimed is:
 1. A system for eye-tracking, the system comprising:a data collection unit that acquires at least one of face information ofa user, location information of a user, user number information, andhuman body information of a user from an image captured by aphotographing device installed at each of one or more points set withina three-dimensional space; an eye tracking unit that estimates an eyegaze location of a corresponding user from the face information of theuser and the location information of the user in the three-dimensionalspace; and an eye-related analysis unit that analyzes eye-relatedcontents of the user based on one or more of the user numberinformation, the human body information of the user, and eye gazelocation information of the user, wherein the eye-related analysis unitanalyzes an eye distribution degree for each object in thethree-dimensional space based on the user number information and theuser eye gaze location information.
 2. The system of claim 1, whereinthe human body information of the user comprises one or more of gender,age, race, and emotion of the user; and the eye-related analysis unitanalyzes eye dwelling time individually for one or more of an age,gender, race, and emotion of a user for a predetermined object in athree-dimensional space based on the human body information of the userand the eye gaze location information of the user.
 3. A system foreye-tracking, the system comprising: a data collection unit thatacquires at least one of face information of a user, locationinformation of a user, user number information, and human bodyinformation of a user from an image captured by a photographing deviceinstalled at each of one or more points set within a three-dimensionalspace; an eye tracking unit that estimates an eye gaze location of acorresponding user from the face information of the user and thelocation information of the user in the three-dimensional space; and aneye-related analysis unit that analyzes eye-related contents of the userbased on one or more of the user number information, the human bodyinformation of the user, and eye gaze location information of the user,wherein the eye-related analysis unit analyzes a ratio of a person wholooked at a predetermined object to a floating population for the objectin the three-dimensional space based on the user number information andthe user eye gaze location information.
 4. The system of claim 3,wherein the eye-related analysis unit determines to change the object toanother object or to change a location of the object in thethree-dimensional space when the ratio of the person who looked at theobject to the floating population is less than or equal to a presetratio for the object.
 5. The system of claim 1, wherein the user humanbody information includes one or more of gender, age, race, and emotionof the user; and the eye-related analysis unit determines whether thereis a harmful object at a point corresponding to a location gazed at bythe corresponding user or whether the point is a preset prohibited zoneor a dangerous zone, using one or more of the human body information ofthe user as a reference.
 6. The system of claim 1, wherein the userhuman body information comprises one or more of the gender, age, race,and emotion of the user; and the eye-related analysis unit extracts oneor more of eye dwelling time and the number of gazing eyes for a presetobject or a preset zone of a corresponding user based on the user eyegaze location information and determines a risk index of a correspondinguser based on one or more of the human body information of the user, theeye dwelling time, and the number of gazing eyes.
 7. A method foreye-tracking comprising: acquiring at least one of face information of auser, location information of a user, user number information, and humanbody information of a user from an image captured by at least onephotographing device installed at at least one point set within athree-dimensional space; estimating an eye gaze location of acorresponding user from the face information of the user and thelocation information of the user in the three-dimensional space; andanalyzing eye-related contents of the user based on one or more of theuser number information, the human body information of the user, and eyegaze location information of the user, wherein the eye-related analysisunit analyzes an eye distribution degree for each object in thethree-dimensional space based on the user number information and theuser eye gaze location information.